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ABSTRACT 

A previous multivariate analysis of variance (MANOVA) 
study of attitudes toward interactive video (ITV) of participants in 
distance education had indicated that generalized variance was not 
homogeneous across all groups. Since heterogeneity of variance in 
MANOVA has previously been shown to change Type I error rates in 
unequal group size, detected differences were suspect. This study 
investigated whether the detected differences in attitude toward ITV 
would be identical if the total participant population with unequal 
group sizes were used or if random sampling of equal size groups was 
conducted. The analysis was conducted by using all participants 
(unequal group size) and by randomly selecting 31 subjects from each 
of 4 groups formed by site (home or remote site) and grade (high 
school or college). Random selection of 31 subjects per group and 
analysis was replicated 4 times. Results support the hypothesis that 
a larger generalized variance associated with a larger sample size 
does producf^ a conservative Type I error rate. These results also 
indicated that if there was no association between sample size and 
generalized variance, the effects of differences in variance will be 
canceled. When, however, the difference in generalized variance was 
less than two times the smaller, the hypothesis that Type I error was 
liberal did not occur. This suggests that there may be a minimum 
difference in variance (and possibly in sample size) to be exceeded 
before Type I error is seriously affected. (Contains three tables and 
five references.) (Author/SLD) 
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Abstract 

A two-way multivariate analysis of variance (MANOVA) based on location (home/remote 
site) and grade (college/high school) had previously been conducted using data produced by 
interactive >ndeo participants in a rural ^palachian area in southwest Virginia. A comparison 
of the generalized variance from this study revealed that the generalized variance was not 
homogeneous across all groups. Since heterogeneity of variance in MANOVA has previously 
been shown to change type I error rates in unequal group size, detected difrerences were suspect. 
The purpose of this study was to determine if the detected differences in attitude toward ITV 
would be identical if the total participant population with unequal group sizes were used or if 
random sampling of equal size groups was conducted. 

This analysis was conducted using all participants (rmequal group size) and by randomly 
selecting 3 1 subjects from each of the four groups formed by site and grade. Random selection of 
3 1 subjects per group and analysis was replicated four times. Results from this study support the 
hypothesis that a larger generalized variance associated with larger sample size does produce a 
conservative type I error rate. These results also indicated that if there was no association 
between sample size and generalized variance, the effects of differences in variance will be 
canceled. When , however, the difference in generalized variance was less that two times the 
smaller, the hypothesis that type I error was liberal did not occur. This suggests that there may be 
a miiumum difference in variance to be exceeded before type I error is seriously affected and 
possibly a minimum difference in sample size. 
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Results of an Interactive Video Attitude Survey when Using the Total Group with Unequal Group 
Sizes versus Five Random Samples with Equal Group Sizes 

Increased use of interactive video (ITV) fiar distance learning students has made classes 
previously inaccessible to rural high school students available locally. College courses are also 
now offered at remote sites by interactive video permitting students to take the class without a 
lengthy drive. In some localities home bound students may now partake of classroom interaction 
by ITV. Thus, interactive video may be perceived as a means of providing equal educational 
opportunities to all students. 

While interactive video may provide access to equal educational opportunities in rural areas, 
there is an increasing need for research into and evaluation of the effectiveness of ITV programs. 
Although ITV technology has advanced rapidly in recent years, there is increasing evidence that 
no one technology works in every application (Linking, 1989). In addition, since effectiveness of 
any prog ram is related to a participant’s perspective toward that program, attitude assessment is 
an ftS!s<»Titial component in evaluating a program. Assessment of participant attitudes is conducted 
by surveying the total current population as a representative sample of all participants (future or 
past) had they been currently enrolled, or sampling specific components of the current population. 
Using the total enrollment may produce unequal sample sizes with results which heterogeneity of 
variance may confound. Sampling for specified characteristics yields equal sample sizes, but may 
resuh in a reduction of power to detect differences. 

A two-way multivariate analysis of variance (MANOVA) based on location (home/remote 
ate) and grade (coUcge/high school) was conducted on data produced by interactive video 
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participants in a rural Appalachian area in southwest Vir^nia. A comparison of the generalized 

variance from this sUidy revealed that the generalized variance was not homogeneous across all 
groups. The purpose of this study was to determine if the detected differences in attitude toward 
rrV would be identical if the total current participant population with unequal group sizes were 
used or if random sampling of equal size groups was conducted. 

Literature Review 

Prior research by Holloway and Dunn (1967) revealed that the type I error rate changes due to 
heterogeneity of variance when using multivariate analysis of variance with unequal sample sizes. 
When the generalized variance of the larger group was three times that of the smaller group, the 
type I error rate decreased providing a more conservative test of the hypothesis (ie, dependent of 
difference in size of the two groups, type I error of .05 detected at a .01). When the generalized 
variance of the larger group was increased to 10 times that of the smaller, this effect was more 
pronounced. Using seven dependent variables, the type I error (.05) changed to .00 with group 
sizes of 15 and 35. Conversely when the larger generalized variance was associated with the 
smaller group, the type I error was liberal (ie, alpha of .05 detected at .09 with a magnitude of 
difference of 3). When the larger generalized variance was increased to 10 times the smaller 
variance, if the larger variance was associated with the smaller group type I aror of .05 became 
.24. 

Hakstian, Roed and Linn (1979) supported this finthng but also added that severely unequal 
group size can produce large changes in the type I error even in cases of mild heterogeneity. 
They also deihonstrated that the type I error is conservative if the larger variance is associated 
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with tiie larger group and liberal if the larger variance is associated with the smaller group. 

Stevens (1986) has added that if Box’s M test for homogeneity of variance is significant whh 
unequal group aze, Hotellings trace is liberal if larger variance is associated with the smaller 
group and conservative if larger variance is associated wit*i the larger group. He further adds if 
the Box’s M test for homogeneity of variance is significant with equal groups size, then type I 
error is only slightly affected and ifi in multiple groups, there is no association between variance 
^nd sample size, effects on type I error due variance differences will by group size will be 
cancded. 

These findings lead to the following hypotheses for this study . 

1. If variance is not associated with sample size, detected differences will be similar in the 

unequal and equal size samples. 

2. If the larger variance is associated with the larger sample size significant differences 
detected in the total sample will also be detected in the equal size samples 

3. If the larger variance is associated with the smaller sample sizr* significant differences 
detected m the total sample will not be detected in the equal size samples 

Method 

All students enrolled in an interactive video class at a southwest Virginia facility during the 
Spring semester, 1995, were surveyed. Surveys were administered during the regularly scheduled 
class time by the class instructor or remote facilitator. One hundred eighty-four student surveys 
were returned. One ofthe returned surveys was unusable. Analysis was conducted using the 
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remaining 183. The sample was composed of 106 high school students (66 home ate, 40 remote 
site) and 77 college students (46 home site, 3 1 remote she). 

The survey instrument consisted of demographic information about the participant and 24 
Likert type questions concerning the ITV program answered by all participants. Prior exploratory 
factor analysis on this quesdonn^e has suggested seven constructs which explain 64% of the 
variance for the 24 objective questions shared by both remote and home site students. Constructs 
established in this model consisted of ITV Program Evaluation accounting for 25% of the 
variance. Materials Support, Class Evaluation, Audio, Environment, Student Behavior, and 
Student Interaction. 

Factor scores for each subject were obtained by multiplying the standardized score for each 
variable by the factor score coefficient (Norusis, 1988). If the variable was a missing value, the 
mean (0) was imputed. Factor scores were then used in a two way multivariate analysis of 
variance (MANOVA) in SPSS/PC+ to determine if there were statistically significant (p<.05) 
differences in the seven constructs forming this model by she or grade. If significant main or 
interaction effects were detected, the constructs contributing to significance were also determined. 
When detecting significance of contribution for the seven univariate constructs, the alpha level 
was changed to a<.01 . This analysis was conducted using all participants and by randomly 
selecting 3 1 subjects firom each of the four groups formed by site and grade. Random selection of 
3 1 subjects per group and analysis was replicated four times 

Results 

When grouped by site and grade, their was no association between variability and sample 
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dze (see Table 1). In this instance, a statistically significant interaction effect (p<.0S) by grade 
and she was detected in ad samples.. Class Evaluation and Audio contributed significantly (<.01) 
in the unequal group size sample. The audio construct contributed significantly (pSOl) to the 
interaction effect in aQ five equal ^e samples. The class evaluation and student behavior 
constructs contributed significantly to the interaction effect in only one of the five samples (see 
Table 1). 



Insert Table 1 about here 



The generalized variance for high school students (n=l06) was 10 times larger than the 
variance of college students (n=77). A statistically significant (p<05) main effect of grade was 
detected in all samples. The materials support construct contributed significantly to the main 
effect of grade in the unequal size sample and the equal size samples, while the student behavior 
construct contributed in the unequal size sample and four of the five equal size samples (see Table 
2 ). 



Insert Table 2 about here 



The generalized variance for remote site students (n=71) was 1.6 times the variance of home 
site students (n=l 12). Both materials support and audio contributed to the significant main effect 
of site in the unequal size sample and one of the equal size samples. In the other equal size 
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sample with a significant main effect of site, only multivariate significance was detected. No 
individual contributor was significant (see Table 3). 



Ins.-^ Table 3 about here 



Discussion 

When the generalized variance did not vary consistently with sample size, a significant 
interaction effect was detected in the unequal size groups. Detection of a significant multivariate 
interaction effect in all five randomly selected samples supports the total group findin g of a 
multivariate interaction effect and hypothesis I . There was, however, a discrepancy between 
other contributors. One of the equal size samples also detected the student behavior construct as 
a signifirant contributor. TMs construct was not statistically significant in the imequal group size 
sample or in the other four equal size samples. This specific instance probably reflects sampling 
variability. In addition, the class evaluation construct was detected as a significant contributor in 
the unequal group size sample and one equal group size sample. 

It was also hypothesized that if the generalized variance was larger for the larger group since 
NriANOVA procedures are conservative, significant differences detected in the unequal group size 
sample would also be detected in the equal group size sample. This hypothesis was supported by 
the fiictor grade. In this instance, the larger variance was associated with the larger group. A 
statistically si gnificant main effect was detected in all samples. The Materials Support construct 
was a significant contributor to multivariate significance in each sample. Student behavior was a 
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significant contributor in all samples except one. 

Statistically significant main eflfects were also detected in the unequal size group sample for 
site. The larger generalized variance was associated with the smaller group size for this factor. 

As hypothesized, statistical significance was not detected in three of the equal size samples. 
Significance was, however, detected in two of the equal size samples. In one of the significant 
equal group samples and the unequal group sample, the contributors to multivariate 
rignificance were materials support and audio. In the other equal size group sample, only 
mul tivariate significanc e was detected. This suggests that the varijd)ility difference between sites 
(1.6 times) may not have been sufficiently different to cause the hypothesized results. 

Conclusion 

Results fi-om this study support the hypothesis that a larger generalized variance associated 
with larger sample size does produce a conservative type I error rate. These results also indicated 
that if there was no association between sample size and generalized variance, the effects of 
differences in variance will be canceled. 

When , however, the difference in generalized variance was less that two times the smaller, the 
hypothesis that type I error was liberal did not occur. This suggests that there may be a nunimum 
difference in variance to be exceeded before type I error is seriously affected and possibly a 
ftiinimiim difference in sample size. Further research is needed to determine what these minimum 
differences may be. 

It is suggested that researchers using multivariate analysis of variance examine the difference 
between the generalized variances for groups. If larger variance is associated with larger group 
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they may feel relatively certain that type I error is conservative. If larger variance is associated 
with the smaller groups, examine the magnitude of difference. If less than 1.6, the effect should 
be minor. 11^ however, the magnitude of difference is larger than 1 .6, type I error may be inflated. 
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